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A Reminder of our Approach 
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We used our five-step experimentation process to explore this opportunity 


SETTING THE STAGE 


IN FIELD RESEARCH 


IMMERSIVE IDEATION 


DESIGN SPRINT 


WHAT'S NEXT? 


What do we want to 
achieve? 
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What is the internal and 
external context? 


What problems do we 
want to solve and what 
are some potential 
solutions? 


What is the minimum 
viable solution and what 
do users think about it? 


PLAN 


How do we scale? 
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TCPs spend a lot of time inquiring about the importation process and 
managing their accounts (1/3) 


BIS Enquities (Jan - Dec 2017) 


Total number of BIS commercial 


enquiries requiring human 
interaction: 235,000 
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TCPs spend a lot of time inquiring about the importation process and 
managing their accounts (2/3) 


TCCU Enquiries (July 2017 - 2018) 


Total number of TCCU enquiries: 


e 28,000 
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TCPs spend a lot of time inquiring about the importation process and 
managing their accounts (3/3) 


ARL Support Enquiries (July 2017-2018) 


Total number of ARL enquiries: 
713,000 
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CBSA spends significant resources addressing stakeholder enquiries 


:235,137 


Total number of commercial BIS calls 
and emails requiring human 
interaction in the last 12 months 


12,859 


Total number of FASP enquiries over 
the last 12 months 


| 82 FTE 


Total FTE handling commercial BIS 
calls and emails 


9 FTE 


Total FTE handling enquiries in the 
TCCU group 


8 FTE 


Total FTE handling enquiries in the 
FASP group 


$1-2M 


Total possible savings per year 


27,693 


Total number of TCCU enquiries over 
the last 12 months 


$225 — 450K 


Total possible savings per year 


$200 - 600K 


Total possible savings per year 
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TCPs are focused on understanding the importation process and the 
transactions being posted to their account(s) while client service teams 
spend significant time on high volume, low complexity questions 


General 
information 


TCPs want to know about 
various aspects of the 
importation process, such as 
tariff classification, import 
accounting, requirements and 
restriction of importing 
commercial goods taxes and 
duties, among others. These 
questions are in high volume 
and are often generic and FAQ- 
type questions. 


“What OGD permits 
do | need to import X 
commercial items 
into Canada?" 
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Status of 
requests & 
account info 


TCPs want to know about the 
status of their import requests, 
forms (e.g., B3 rejects), 
payments, refunds or help with 
issues with EDI production and 
BN mis-entry. TCPs want to be 
updated on information they've 
submitted to CBSA for review or 
action. More transparency for 
end users would minimize 
enquiries to CBSA. 


“What is the status of 
my import request?" 


Statement of 
account & daily 
notice 


Many importers are simply 
looking to understand the status 
of their account. There is 
complexity around getting 
immediate access to their 
account to track amounts owed 
and payments due. Greater 
functionality for end users would 
allow for faster access to their 
account information and less 
enquiries to CBSA. 


^We did not receive 
the statement of 
account for BN#X. 

Please advise on the 
status of the SOA.” 
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Improve the service experience for trade chain partners 


A chatbot results in a better user experience by: 
Allowing immediate access to answers for users 
Facilitating an easy interaction through natural language in both English and French. 


Creating an integrated user support channels that allow for seamless transition between agents or 
teams 


Providing a consistent user experience during each interaction 


Chatbots allow organizations to better serve users o of client interactions will be 
through a user-centric platform that allows them to & D % managed without a human 


Gartner listen to users and better meet their needs. by 2020 
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Reduce the amount of time that trade chain partners spend 
communicating with CBSA 


Swedbank introduced a human-like, conversational AI tt 
Vah O of customer enquiries were 
chatbot with self-serve capabilities to respond to user - 
inquiries O resolved on first contact 
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Reduce the call & email volume, and know when to escalate to humans for 
help 


A chatbot will help trade chain 
partners get things done faster 
than they do today, but there 
will be times where the request 
from the end user is not 
something the bot is capable of 
handling. In these situations 
the bot must escalate to a 
human agent. 


Bradesco introduced an Al chatbot in 2014 to assist with O of help desk calls were 
internal employee queries, and they were diligent in O 


managing, refining and scaling it’s response capabilities eliminated within 2 years 
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SERVES PEST O KONTACT 


For the CARM program one 
of the biggest new se _ S 


I UI 


Me 
Se 


capabilities being delivered = vet 
is an online portal for trade 

chain partners to interact 
with. A chatbot would be a 
logical extension of this 
portal to allow trade chain 
partners to quickly access 
disparate information and 
perform various 
transactions with the 
agency without navigating 
through pages 
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A chatbot could interact directly with trade chain partners through their 
mobile devices 


Along with integrating into the CARM portal, a chatbot could interact with trade chain partners directly through 


their mobile devices for instant access while on the go and provide push notifications from CARM to trade chain 
partners. : 
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A chatbot can be deployed across several different platforms in order to 
best meet importer/ broker needs 


Chatbots will be used more when they are made available in as many places as possible. With minimal 
deployment effort the same bot can live in many different platforms at the same time. 
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The chatbot proof of concept will demonstrate the value and impact of the 
solution 


e 
E. 
< 
(D 


integration of the chatbot into the CARM 
backend 

e Embedding of chatbot into the conceptual CARM 
portal 


e Setup of 1 non-production environment to host the 
proof of concept (POC) 

* Creation of mocked data which is based on (masked) 
production data 

e Configuration of a single scenario used by front line 
staff into chatbot tool (based on interviews with BIS, 
TCCU and FASP) 

e Demonstration of chatbot POC to key CARM 
stakeholders and — provide opportunity to address 
technical / business related questions that surface from 
the POC 

* Creation of executive summary to document key 
findings, outcomes, risks, issues, and next steps 


ts o aoc. ns zo ds So petra 
e uM cet cee crea M ce ce M 2 Ut ne 


pease tos potas 
SSSR ESS 


Provide Business Context Participate in Meetings Support Testing of POC 

Provide background and context Delegate key stakeholders to act as Provide staff to assist with testing 

around the selected user scenario so participants in workshops / working the solution from a business end 

that project team can properly train groups to identify relevant questions users perspective and identify 

and configure the solution to meet to ask the chatbot, confirm user potential issues / defects to 

expected results. scenarios, and make decisions on address. Validate the accuracy of 
behalf of CARM. various documents being produced 
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The proof of concept will adopt Agile principles in order to deliver value in 
a timely manner 


a 


Key Deliverables 
Configured and Trained POC Solution for knowledge Articles / Workflows In-Scope 
Documentation of POC User Scenarios / Questions to Prove 
POC Test Cases & Result 


High Level Analysis of Full Implementation Effort and Approach 
Executive Summary for POC Outcomes, Key Findings, and Next Steps 
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The benefits of the chatbot are aligned closely with the core CARM 
mandate 


0 

= Sin n ae i m I. dene Ms itimat The chatbot will enable TCPs to get 

= importation declaration and release processes for legitimate answers to their questions quickly 

Q trade, by leveraging new technology and reducing anu dcn 

z administrative burden at the border y 

c 

A. 

- Improve client identity management and commodity The chatbot will be able to answer 

- identification. Enhance the identification of Trade Chain authenticated questions, and has 

A Partners (TCPs) and imported goods to improve pre-arrival potential to be leveraged to improve 

- risk assessment and targeting activities pre-arrival activities 

O 
MODERN - A fully digital TRANSFORMED - A re- INNOVATIVE - A capability 
commercial continuum engineered, highly efficient to respond to evolving client 

n incorporating self-service business operation enabled and marketplace needs and 

2 functionality for clients by technology trends 

U 

LLJ 

m 

tà 

O 

= 

fa 

d Users can seek information The chatbot will significantly According to Gartner research, 
from one source, and inquire reduce effort required for the approximately 85% of client 
about both authenticated and three current CBSA contact interactions will be managed 
non-authenticated inquiries options (BIS, TCCU, and FASP) without a human by 2020 
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In order to move the chatbot POC to the next phase, a few key steps need 
to be completed 


* Gain approval for resources to move forward with POC 
e Establish POC team, including relevant CBSA SME(s) 
e Confirm relevant questions and scenarios to prove 


* Setup chatbot environment 
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CBSA - Released under the Access to Information Act 
ASFC - Divulgation en vertu de la loi sur l'Accès à l'information. 


Appendix 
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CBSA - Released under the Access to Information Act 
ASFC - Divulgation en vertu de la loi sur l'Accès à l'information. 
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DRAFT 


BIS data (1/4) 


TCPs spend a lot of time inquiring about the importation process and 


managing their accounts (1/3) 


BIS Enquities (fan « Dec 2017) 


Total number nf BIR conan a 
Bi colis requirinc interactic 
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Based on Suzanne's input on the Workflow Action Activity Report (WAAS 
with Summary and Details Jan — Dec 2017 Monthly.pdf), summarized the 
number of monthly calls received by BIS for business information 
(commercial) related inquiries. The summarized data included the total 
number of calls for each option within the business information menu. 
Determined that a total number of 340,481 calls were received by BIS for 
commercial inquires, between January and December 2017 (note: this 
340,481 does not distinguish between calls where human interaction is 
needed vs not) 


Based on the BIS Phone Monthly Stats file (BIS Phone Monthly Stats Fiscal 
Year 2016 - 2017 2017 - 2018 v3.03.xlsx), it was determined that BIS 
received a total of 657,788 in 2017 including both traveler and commercial 
inquiries 


Based on the above data, it was determined that roughly 5096 of BIS calls 
are commercial inquires (340,481 / 657,788 = 50%) 


As per the BIS Phone Monthly Stats file, BIS treats a total of 303,336 calls in 
tier 1, and 79,336 in tier 2. Both require CBSA employees interaction. 
Assuming that 5096 of the tier 1 tickets are commercial, and 9096 of tier 2 
are commercial (based on a discussion with Suzanne where she mentioned 
that most tier 2 tickets are commercial), it was determined that BIS treats 
approximately 151,668 commercial tickets in tier 1, and 71,402 in tier 2 for 
a total of 223,070 tickets requiring CBSA employees interaction 
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DRAFT 


BIS data (2/4) 


—Á ° The other category includes the following categories (with their related % of 
TCPs spend a lot of time inquiring about the importation process and total volum es) . 
managing their accounts (1/3) 


BIS Enquities (Jan - Dec 2017) O CSA, FAST, PIP, API/ PNR (7.7%) 
o The Administrative Monetary Penalty System (AMPS) (0.5%) 
o Commercial Transporter Obligations (0.6%) 


o  eManifest (3.196) 


uf Siu comucrcial 
Penran irr actioni — i 
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DRAFT 


BIS data (3/4) 


CHA A sp enda significant resources aire 


Total commercial calis recquirinc phumen. interac tion 


65 FTE} 


o handie the cals (assuming 252 


Total possibie savings 


Total commercial calls. ang emails 
5 (assuming 10 min 7 email} 


of wee a oun 
Totali Dos dps savings 
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Based on the BIS E-Mail Monthly Statistics file (2017-18 BIS E-Mail Monthly 
Statistics), it was determined that BIS received a total of 22,754 emails in 
2017. Assuming that commercial inquires represent 50% of Tier 1 emails, 
90% of Tier 2 emails and 95% of HQ emails (% based on conversation with 
Suzanne), it was determined that a total of 12,067 commercial emails were 
received by BIS in 2017 


Assuming that 100% of calls and emails can be answered by a chatbot 
o 235,137 calls and emails are handled yearly 


o 2,351,373 minutes are required to handle the questions (235,137 
calls/emails * 10 minutes per call/email). (Assumed 10 minutes per 
call) 


o 5,225 days are required to handle the questions (2,351,373 minutes / 
60 minutes / 7.5 hours of work per day) 


o 163 days are required per FTE to handle the questions (5,225 / 32 FTE) 
— 65 FTEs is the total FTE (commercial + traveler) based on 
conversation with Suzanne. Assumed that 5096 of them work on 
commercial questions only 


o  20.7FTEs are required to handle the questions in total (5,225 days 
required to handle the questions / 252 working days per year) 


o Potential savings of $2,073,520 if 100% of the calls and emails can be 
answered by a chatbot 
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DRAFT 


BIS data (4/4) 


PUTixmes: cadum uus ud sy à iz Qvi kW Sn NEN Sogn) Cpu" anne 
CIA spends signilicant resources address 


O 
O 
O 
O 
O 
wiring human —— — 
L10minpercaib 
| Days of commercial calls per FTE (Assuming 
165 FTE) O 


| Total FTE required to handle the calls 
{assumir ays of work mg year 
i Total possible savings 


i Total commercial emails 
aiio (assuming 10 min / email) 
d Ci emalis per FTE 
i Total FTE required to handle the emails 
i (assuming 252days of work a year) 
es 

EURE 


calis and emails 
is (cssuminc 


40 min / 


and emails per FTE z 


He the cols and 
ETE nae 36. 
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° Assuming that 50% of calls and emails can be answered by a chatbot 


117,569 calls and emails are handled yearly 


1,175,686 minutes are required to handle the questions (235,137 
calls/emails * 10 minutes per call/email). 


2,613 days are required to handle the questions (2,351,373 minutes / 
60 minutes / 7.5 hours of work per day) 


82 days are required per FTE to handle the questions (5,225 / 32 FTE) — 
65 FTEs is the total FTE (commercial + traveler) based on conversation 
with Suzanne. Assumed that 5096 of them work on commercial 
questions only 


10.4 FTEs are required to handle the questions in total (5,225 days 
required to handle the questions / 252 working days per year) 


Potential savings of $1,036,760 if 5096 of the calls and emails can be 
answered by a chatbot 
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CBSA - Released under the Access to Information Act 
ASFC - Divulgation en vertu de la loi sur l'Accès à l'information. 
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CBSA - Released under the Access to Information Act 
ASFC - Divulgation en vertu de la loi sur l'Accès à l'information. 


DRAFT 


TCCU data (1/2) 


Hx qun enims : : Based on the data provided by Rick Stark, we took the total number of 
TCPs spend a lot of time inquiring about the importation process and enquiries from the ECU (26 j 143) and TSU (1 ,550) teams, which have us 
APACHE EE EESE 27,693 total enquiries to the TCCU team between July 2017-2018 
TCCU Enquiries (fuly 2017 - 2018) 
° The other category includes the following issue categories (with their 
related % of total volumes): 


o Application (1.6%) 
o Barcode testing (0.4%) 
o Bulletin request (0.2%) 


o Distribution list (1.0%) 


o  ECCRD request (1.3%) 
o EDI account (0.9%) 

o Other (2.1%) 

o Outage (0.4%) 

o Paper request (1.3%) 

o Portal account (2.3%) 
o Set aside (0.004%) 


o Testing (.9%) 
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DRAFT 


TCCU data (2/2) 


CIA spends signilicant resources adde 
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Ye Ro aX xc m ios go E SS Fe 6D 
ARIS SURE SRE ISS BESI ASSA BES 


Based on a discussions, it was determined that 9 FTEs work in TCCU 
answering calls / handling tickets 


Assuming that 50% of calls and emails can be answered by a chatbot 
oO 4.5 FTE time could be fulfilled by a chatbot 


o Assuming an annual salary of $100,000, the total savings by a chatbot 
is approximately $450,000 


Assuming that 2596 of calls and emails can be answered by a chatbot 
o 2.25 FTE time could be fulfilled by a chatbot 


o Assuming an annual salary of $100,000, the total savings by a chatbot 
is approximately $225,000 
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DRAFT 


FASP data (1/3) 


TCPs spend a lot of time inquiring about the importat 
managing their aceounts (3/3) 


ion process and 


ARL Support Enquiries {luly 2017-2018] 
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Total tickets/enquiries were provided by Barbara in a screenshot of a pie 
chart, providing us with the types of enquiries (FASP labels) and the volume 
of tickets per label 


KBAs were removed (based on Aran and Barb's recommendations). Note: 
some of the "other" category may include KBAs, given that we were only 
provided with the first 39 rows of the table. 


The total from the pie chart components was used as the total number of 
enquiries (representing the number of labels, assuming this is the number 
of issues). 


The ‘other’ row in the raw data is calculated by taking the total from the pie 
chart, subtracting the KBAs, then subtracting the total of the remaining 29 
rows provided. 


"Credit" label FASP label as divided into two (half allocated to each 
category) to Payment and Credit & Disbursement since we were told by 
Barb that these enquiries could fall under either 


The original FASP labels were assigned parent labels to bubble up to a 
higher level — refer to table categorizing 
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DRAFT 


FASP data (2/3) 


° The original FASP labels were assigned parent labels to bubble up to a 
her level — refer to table categorizing: 


TCPs spend a lot of time inquiring about the importation process and hi 
managing their aceounts (3/3) 


ARE Support Enquiries [uly 2017-2018] | BN Change BN Change 


| Total number enmgieihes: | 


Payment Payment 


SOA, DN & Account 
| Overview Document Status 
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DRAFT 


FASP data (3/3) 


: 8 FTE working in the FASP group answering enquiries — provided by Barbara 
skehoider enquiries an d C hri 5 


CIA spends signilicant resources adde 


Assuming that 7596 of calls and emails can be answered by a chatbot 


o  6FTEtime could be fulfilled by a chatbot 


o Assuming an annual salary of $100,000, the total savings by a chatbot 
is approximately $600,000 


Assuming that 2596 of calls and emails can be answered by a chatbot 


o 2 FTE time could be fulfilled by a chatbot 


o Assuming an annual salary of $100,000, the total savings by a chatbot 
is approximately $200,000 


io0,000.00 
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Helios 
Al 4 Leg, Regs, Policies 
Prototype Tool | 


Business Case 
August 2019 


CBSA Assessment and 
Revenue Management 


CARM 
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Current vs. Al-powered Workflow 


Al-powered A + = 


i 


Current | 
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AI-POWERED 


CURRENT . 


Business Case 


As the Al prototype tool is a new innovation, we will measure its effectiveness across a 
number of dimension by comparing the current method to an Al-powered method. 


^ 


# OF DOCUMENTS 


TIME SAVED 


!] 
| 


jo 


USABILITY 


NEW TOOLS 


e ~750 total 
documents ingested 

* Instant search across 
all documents in real 
time 


Advanced search ° 
capabilities and time saved 
An example complex 

query took 35 minutes for 
search, analysis, clean, and 
extraction (multiple 
keywords overlaid) across œ 
entire document set 


Higher accuracy, and 
resulting confidence 
on initial search 
results and 
completeness of 
analysis 

Better contextual 
insights 

Advanced search 
capabilities 


Simple, easy to use 
interface 

Makes policy 
documents easy to 
peruse 

Both textual and 
visual search 
available 

Sharing 


Visual Connectivity 
Graph 

Thematic Clouds (by 
topic, by owner) 
Outdatedness 
Prescriptivity 
Readability 

Faster workflow & 
collaboration 


e User based manual 
or binary search by 
paper or justice 
website 

Scan one document 
at a time 


Search one key word, . 
one document at a time 
Based on experience, the 
complex query example ° 
would take more than 

one day to complete 

(13x more time) ° 
Significant time saving of 

~ 200 days anticipated 

based on current usage 
volume (300 

queries/year). See 

appendix for 


Accuracy based on 
accuracy of the initial 
search 

No contextual 
insights unless 
provided by SME 

No adv. search 
capabilities. SME's 
are used to narrow 
down areas to search 


User searches 
paper, books, or on 
the Justice website 
New users would 
struggle with search 
capability 

Search is strictly 
basic binary search 
only. 


Users would have to 
custom build new 
tools using commonly 
available tools such 
as MS excel or 
access. 

Limited collaboration, 
typically by paper or 
by emails 
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Expected Benefits 


© ZUU days 


SUU queries 


* Estimated based on 
current usage volume 
of 50 queries over 2 
months 


e 50% simple and 50% 
complex queries 


e 13x to 18x time saving 
per query based on 
real life examples 


... allowing policy analysts to focus on the core, value-added analysis 


CARM CBSA Assessment and 
Revenue Management 


of time saved on searching tasks per 
year based on... 


O 
ove Bs 


Single user Leg/Regs only 


* Based on today's * Based on time it takes to 
focused user group of search through legislative 
CARM Program and regulatory documents 
Authority (143 documents in Helios 


currently) 


Additional benefit to 
be realized when 


Significant additional 


rolled out to broader benefit to be realized when 
user base (e.g., CARM d-memos (additional 369 
project team, strategic documents), forms, customs 
policy branch, etc.) notice are considered 
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What Could Be Next for Helios? 


——— 
P une es 


| IMPROVED EXPANDED EXPANDED | 8$ // BROADER 
USABILITY DATA ANALYTICS USE ` —.. ROLLOUT 
| CASES a 
| * Tagging multiple | : e Additional | : e Public sentiment | : e Multi-user 
| results of a search | | legislation and E analysis E workflow support 
+ Reporting features | | regulations ||* Entity-centric || * User security and 
| and frequently —— | | | connection graph | | access controls 
| used template * Trade Chain | visualization | | e Bilingual support 
| e Auto highlight | : PAINEL | : e Visualization of D 
| certain outdated | |. CBSA internal || connectivity at 
terms, prescriptive | | data sources || section level 
terms, and other |. | 
agencies 
CARM | seen j 
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Discussion 


o How would you like to proceed with Helios? 


Would the team like to continue using the tool? Could other areas 
within CBSA benefit from the tool? 


CARM CBSA Assessment and 6 
Revenue Management 
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Appendix — Expected Benefits Assumptions 


1G 


CURRENT 


EODEM 35 minutes » Full day 
advanced query 


- Annual search volume of 300 
Annual search volume (based on current usage of 50 searches per 2 months) 
- Assume 150 simple / 150 complex queries 


ITEM 


Expected time spent on 113 hours 1,575 hours 
search per year (^15 days) (7210 days) 
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